Medical imaging device, method, and use

ABSTRACT

A medical imaging device, comprising a illumination unit, can be configured to illuminate a tissue area with light from a first spectral range, comprising a range of visible wavelengths, and with light from a second spectral range, which is different from the first spectral range, an imaging unit, configured to detect light from the first spectral range and to generate a first image of the tissue area on the basis of the detected light from the first spectral range and furthermore configured to detect light from the second spectral range and to generate a second image of the tissue area on the basis of the detected light from the second spectral range, and a superposition unit, configured to generate a superimposed image on the basis of the first and second image in such a way that, in the superimposed image, on the basis of a visual highlighting, it is possible to identify whether and where the tissue area comprises highlight regions which are characterized by an increased emission of light from the second spectral range in comparison to other regions of the tissue area. The document furthermore relates to a method for image-based support for a medical intervention, and to a use of an imaging device in such a method.

CLAIM FOR PRIORITY

This application claims the benefit of priority of German ApplicationNo. 10 2019 217 541.4, filed Nov. 13, 2019, which is hereby incorporatedby reference in its entirety.

TECHNICAL FIELD

This document relates to a medical imaging device, to a method forproviding image-based support for a medical intervention, and to a useof such an imaging device in such a method.

BACKGROUND

During medical interventions or examinations, it is often necessary todistinguish between different types of tissue within a single tissuearea. For example, it may be necessary to distinguish betweendefectively altered or proliferating tissue (abnormal tissue) andsurrounding, healthy tissue of the tissue area. Based on such adistinction, medical staff, for example, are able to make a diagnosisand/or decide on a site for application of a local therapy, for examplean injection or excision of the abnormal tissue. The abnormal tissue maybe, for example, tumor tissue, cyst tissue and/or an expanding tissuegrowth, for example cholesteatoma tissue, i.e. an inflammatory expandingtissue growth in a middle-ear tissue area.

SUMMARY/OVERVIEW

Medical imaging devices by means of which images of a tissue area may beproduced may be consulted in order to distinguish between differenttypes of tissue within the tissue area. Here, the problem may arise thatthe different types of tissue are not always visually clearlydistinguishable. For example, cholesteatoma tissue is not easilyvisually distinguishable from surrounding middle ear tissue, inparticular bone tissue. In particular, this hinders complete excision ofthe abnormal tissue, something which is indispensable for successfultherapy.

The proposed imaging device can allow an improved distinction of varioustypes of tissue within a tissue area. Furthermore, a method forimage-based support for a medical intervention and a use of the proposedimaging device in the proposed method are provided.

A medical imaging device in an example comprises

-   -   an illumination unit, configured to illuminate a tissue area        with light from a first spectral range, comprising a range of        visible wavelengths, and with light from a second spectral        range, which is different from the first spectral range,    -   an imaging unit, configured to detect light from the first        spectral range and to generate a first image of the tissue area        on the basis of the detected light from the first spectral range        and furthermore configured to detect light from the second        spectral range and to generate a second image of the tissue area        on the basis of the detected light from the second spectral        range,    -   a superposition unit, configured to generate a superimposed        image on the basis of the first and second image in such a way        that, in the superimposed image, on the basis of a visual        highlighting, it is possible to identify whether and where the        tissue area comprises highlight regions which are characterized        by an increased emission of light from the second spectral range        in comparison to other regions of the tissue area.

With the imaging device according to the present approach, by thegeneration of images (specifically the first and second image),different types of tissue within the tissue area may be clearlydistinguished from one another on the basis of the detected light fromdifferent spectral ranges (specifically from the first and secondspectral range) in conjunction with the visual highlighting in thesuperimposed image.

In particular, this concerns types of tissue that differ from oneanother with regard to the spectral properties within the first and/orsecond spectral range, in particular the second spectral range. Theincreased emission of light from the second spectral range in thehighlight regions in comparison to the other regions of the tissue areamay occur on account of increased reflection and/or scattering and/orremission and/or other emission of light from the second spectral rangein comparison to the other regions of the tissue area.

The superimposed image produced by means of the superposition unit, atthe same time, serves advantageously as an overview of the tissue area,on the basis of which various features of the tissue area which resultin particular from the first image may be identified, and also serves tomake use of additional information contained in the second image as aresult of the visual highlighting, such that it is possible to identifywhether and where the tissue area comprises highlight regions. Theinformation thus provided may enable or simplify diagnoses and/or andtherapeutic decisions.

In this application, “highlight regions” include both the aforementionedregions of the tissue area with increased emission of light from thesecond spectral range in comparison to other regions of the tissue area,as well as the regions of an image corresponding to these tissueregions, in particular the regions of the superimposed imagecharacterized by means of the visual highlighting.

The term “visible wavelengths” is understood to mean wavelengths ofvisible light, that is to say wavelengths approximately between 380 nmand 750 nm, in particular between 400 nm and 700 nm. The first spectralrange and/or the second spectral range may include a single continuousinterval of wavelengths or a plurality of intervals of wavelengths.

The second spectral range may be contained wholly or partially in thefirst spectral range, may overlap the first spectral range in part, ormay be disjoint from the first spectral range, that is to say has nooverlap therewith. The second spectral range may preferably have a lowerbandwidth than the first spectral range. The first and/or secondspectral range may also comprise wavelengths outside visible light, inparticular wavelengths in the near-ultraviolet range and/or in thenear-infrared range. The first spectral range may comprise, for example,all visible wavelengths, inclusive or exclusive of the second spectralrange, or a sub-range of the visible wavelengths, inclusive or exclusiveof the second spectral range.

The first image and/or the second image and/or the superimposed imagemay be an optical image, that is to say a real or virtual imagegenerated or generatable by means of an optical imaging process, adisplay image, that is to say an image presented or presentable on adisplay, screen or another display unit, or an image dataset, that is tosay a digital dataset detected or detectable by means of a detectionunit and corresponding to an optical image and/or a display image.

By means of a continuous and/or repeated generation of the first and/orsecond image and/or of the superimposed image, a change to the tissuearea, for example a change on account of a performed excision or othertherapeutic measures, may be monitored, for example during a medicalintervention, and may be analyzed for further therapeutic and/ordiagnostic steps.

The first spectral range may comprise at least one sub-range of visiblewavelengths greater than 450 nm and/or omit a sub-range of wavelengthsless than 450 nm, in particular at least one sub-range of the secondspectral range. The second spectral range may include at least onesub-range of wavelengths between 350 nm and 500 nm, in particularbetween 370 nm and 480 nm, preferably between 400 nm and 450 nm, and mayomit at least one sub-range of wavelengths smaller than 380 nm and/orgreater than 450 nm.

For example, the second spectral range may consist of a single intervalwith a lower limit between 350 nm and 420 nm and an upper limit between430 nm and 500 nm.

With the spectral ranges thus selected, the imaging device isparticularly well suited for examining a tissue area in which abnormaltissue is characterized by an increased emission of light within theaforesaid ranges (350 nm to 500 nm, that is to say approximatelynear-ultraviolet to blue-green), since such abnormal tissue is thenidentifiable in the superimposed image on the basis of the visualhighlighting. For example, in this way, it is possible to make use ofthe fact that cholesteatoma tissue has an increased reflection andreduced absorption of light with wavelengths from 370 nm to 480 nm incomparison to surrounding bone tissue in the middle ear. Therefore, thevisual highlighting in this case may facilitate the identification (andthus also decisions regarding the removal) of cholesteatoma tissue bymedical staff.

The omission of a sub-range of wavelengths smaller than 380 nm mayreduce or avoid damage to the tissue area caused by ultraviolet light.The omission of a sub-range of wavelengths greater than 450 nm enables agood spectral separation of the first and second spectral range.

A spectral range omitting/not overlapping a wavelength range means,throughout this application, that said spectral range essentiallyomits/does not overlap said wavelength range. In particular, whengenerating/detecting light of said spectral range using a physicalseparation (e. g., by means of an optical filter and/or an emission bandof the illumination unit and/or a detection band of the imaging unit), acertain fraction of the omitted/not overlapping wavelength range maystill be generated/detected. Said fraction may be, for example, lessthan 10%, in particular less than 1%, preferably less than 0.1% of thelight which is generated/detected overall.

The first and/or second spectral range may also be selected differently,such that the imaging device is particularly suitable for distinguishingother types of tissues. For example, the second spectral range mayinclude wavelengths in an interval around a central wavelength of 490 nmand/or in an interval around a central wavelength of 640 nm and may omitat least sub-ranges of wavelengths outside the respective intervals,wherein the intervals may be given by suitable bandwidth, for examplebandwidth between 10 nm and 100 nm around the respective centralwavelengths. The imaging device is therefore well suited, for example,for examining nerve tissue; for example, parotid tissue and nerve tissuehave an increased reflection and reduced absorption at wavelengths ofapproximately 490 nm and approximately 640 nm in comparison tosurrounding tissue.

In order to allow the generation of the first image and of the secondimage, the imaging device may be configured to allow a separatedetection of light from the first spectral range and of light from thesecond spectral range. This may be achieved in different ways, forexample by spatially separate and/or temporally separate detection oflight from the first spectral range and light from the second spectralrange in accordance with the possibilities described hereinafter.

The illumination unit may be configured for sequential illumination ofthe tissue area with light from the first and second spectral range, forexample such that the tissue area is first illuminated with light fromthe first spectral range and is then illuminated with light from thesecond spectral range. Since the temporal sequence of the illuminationwith light from the first and second spectral range is known, atemporally separate detection of light from the first spectral range andof light from the second spectral range is thus possible. By sequentialillumination of the tissue area with light from the first and secondspectral range, it may be ensured, in particular, that spectralproperties of the tissue area in the second spectral range aredetectable separately, i.e. without simultaneous illumination of thetissue area with light from the first spectral range.

For sequential illumination, the illumination units may comprise, forexample, a broadband light source, for example a halogen-metal halidelamp or a broadband LED light source, in each case configured tosimultaneously emit light both from the first and from the secondspectral range. Furthermore, the illumination unit may have a filterunit that is able to be switched over between transmission of the firstand second spectral range.

The illumination unit, however, may also comprise a plurality of lightsources, for example LED light sources and/or laser light sources, whichare able to be switched over and/or combined for the sequentialgeneration of light from the first and second spectral range.

The illumination unit may be configured for the simultaneousillumination of the tissue area with light from the first and secondspectral range. The simultaneous illumination of the tissue area allowsa particularly simple design of the illumination unit, for example withuse of a broadband light source as described above, although in thiscase there is no need for a switchable filter unit.

The imaging unit may be configured for the sequential generation of thefirst and second image. A particularly simple design of the imaging unitis thus possible, and the imaging unit in this case, for example, mayhave a single beam path and a single broadband detector for generationof the first and second image.

In particular, it may be provided that the imaging unit is configuredfor the sequential generation of the first and second image, and theillumination unit is configured for the sequential illumination of thetissue area with light of the first and second spectral range. To thisend, for example, an alternating insertion and/or removal of one or moreoptical filters of the illumination unit and/or a switching of lightsources of the illumination unit may be synchronized with a detection ofa series of images, so that, within the series of images, the first andsecond image or the first and second series of images alternate.

It may also be provided that the imaging unit is configured for thesequential generation of the first and second image, and theillumination unit is configured for the sequential illumination of thetissue area with light of the first and second spectral range. To thisend, for example, an alternating insertion and/or removal of one or moreoptical filters of the imaging unit may be synchronized with a detectionof a series of images, so that, within the series of images, the firstand second image alternate.

The imaging unit may be configured for the simultaneous generation ofthe first and second image, wherein the first and second image arespatially separated by means of at least one optical filter, that is tosay light from the first spectral range and light from the secondspectral range are detectable spatially separately. By means of thesimultaneous generation of the first and second image, a particularlyhigh time resolution is possible, for example.

It may also be provided, for example, that the imaging unit isconfigured for the simultaneous generation of the first and secondimage, and the illumination unit is configured for the simultaneousillumination of the tissue area with light of the first and secondspectral range. To this end, for example, a broadband light generated bymeans of broadband illumination and emitted by the tissue area may beseparated spatially by means of a dichroic mirror into two components,corresponding to the first and second spectral range, and may bedetected by means of separate sensors or spatially separate regions of asingle sensor.

By means of a suitable selection and combination of the differentmentioned possibilities for sequential and/or simultaneous illuminationand/or detection, the various advantages may be combined with oneanother and optimized in respect of a desired application.

The medical imaging device may be or may comprise an ear-nose-throat(ENT) microscope and/or an operating microscope (surgical microscope),in particular a surgical microscope for the ENT area, and/or anendoscope, whereby it is particularly suitable for use in medicalprocedures that are performed with use of such instruments.

The imaging unit may have at least one objective lens for detectinglight from the first and second spectral range. The imaging unit and/orthe superposition unit may comprise at least one eyepiece for visuallydisplaying the first image and/or the second image and/or thesuperimposed image.

The medical imaging device may comprise a detection unit which isconfigured to detect a first pixel dataset, corresponding to the firstimage and/or the first series of images, and a second pixel dataset,corresponding to the second image and/or the second series of images.

The first and second image are provided in the form of a digital imagedataset, specifically in the form of the first and second pixel dataset,by means of the detection unit, which allows for flexible furtherprocessing, storage and/or display.

Such a detection unit may comprise, in particular, at least one imagesensor (in the case of sequential detection of the first and secondimage, preferably at least two image sensors), for example a CCD or CMOSchip, which is configured for spatially resolved detection of the firstand/or second pixel dataset.

The superposition unit may be or may comprise an image processing unitwhich is configured to generate a third pixel dataset, corresponding tothe superimposed image and/or the superimposed series of images, on thebasis of the first and second pixel dataset.

The detection unit may be configured for the repeated detection ofimages with a repetition frequency of at least 30 Hz, preferably atleast 60 Hz. The image processing unit may be configured to generate thethird pixel dataset with a latency of less than 100 ms, preferably lessthan 50 ms.

Due to the time resolution thus attainable and short temporal delaybetween the generation of the first and second image and the generationof the superimposed image, a display of the superimposed image in realtime is made possible, whereby in particular a surgical intervention atthe tissue area may be performed with direct visual feedback.

The imaging device may be configured to show and/or display thesuperimposed image by means of one or more displays and/or by means ofone or more eyepieces.

The use of eyepieces may allow a user intuitive and close access to thesuperimposed image. The use of a display may allow a number of people toview the superimposed image simultaneously, and also offers particularlyflexible display options, by means of which, for example, the visualhighlighting may be seen particularly clearly.

The superposition unit may be configured to generate the superimposedimage by superimposing the first and second image, in particularoptically or digitally.

An optical superposition is a particularly simple possibility forgenerating the superimposed image and is possible, in particular, if thehighlight regions in the second image are clear and the first image isnot too light in comparison to the second image. In order to ensurethis, it may also be possible to soften the first image by means of agray filter and/or to suitably adapt an illumination level of theillumination with light from the first and/or second spectral range.

In the case of a digital superposition, an intensification and/orsoftening of the first and/or second image may be achieved in a simplemanner by linear or non-linear scaling of the first and/or second pixeldataset.

The superposition unit may be configured to generate the superimposedimage by alternate generation and/or display and/or hiding and/orrefreshing of the first and second image.

The frequency of the alternation between the first and second imageand/or the refreshing of the first or second image may be at least 30Hz, preferably at least 60 Hz.

By means of the time resolution thus attainable, a display of thesuperimposed image in real time may again be made possible, whereby inparticular a surgical intervention at the tissue area may be performedwith direct visual feedback.

The image processing unit may be configured to generate a highlightimage on the basis of the second image or on the basis of the first andsecond image, which highlight image comprises the visual highlighting.The superposition unit may then be configured to generate thesuperimposed image by superimposing the first image with the highlightimage. The superimposed image may also be generatable by replacingregions of the first image by the highlight image or by correspondingregions of the highlight image.

The image processing unit may be configured to generate the visualhighlighting and/or the highlight image by means of a threshold valuefor the second image and/or by means of a segmentation of the secondimage and/or by means of a color space transformation and/or a featureextraction and/or an object classification and/or a machine learningalgorithm and/or a neural network.

In this way, the visual highlighting and/or the highlight image may bedetermined robustly and/or may be displayed in a clearly visible manner.

The image processing unit may be configured to select a type of visualhighlighting on the basis of a user input. The visual highlighting maythus be adjustable depending on user preference and/or requiredinformation.

The visual highlighting may comprise a color-based delimitation and/oran edging of the highlight regions and/or marking of the highlightregions by means of at least one symbol and/or lettering. The visualhighlighting may comprise a display of a quantitative feature of thefirst and/or second image, for example by means of a color assignmenttable and/or by means of blended numerical values and/or superimposedcontour lines. The quantitative feature may be, for example, a quotientor other function of pixel values of the second and/or first image. Thevisual highlighting may comprise a marking of an absence of highlightregions, for example by means of a symbol and/or lettering introducedinto the superimposed image. The visual highlighting may be or maycomprise a false color display of the second image.

The superimposed image and/or the first image and/or the second imagemay be a two-dimensional image or a stereo image, i.e. athree-dimensional image composed of a left and a right component. Theimaging unit may be configured to divide light emitted by the tissuearea in order to generate a left and a right component of the first andsecond image.

If the superimposed image is a stereo image, the imaging device may beconfigured to display the superimposed image by means of two eyepiecesand/or to show the superimposed image on a 3D display.

Generation of the superimposed image and/or the first and/or secondimage as a stereo image is particularly advantageous for use of themedical imaging device in conjunction with a surgical intervention,since spatial perception is made possible as a result of the stereoimage. Similar advantages are provided if the visual highlighting is ahighlighting in an augmented reality display (AR display).

The detection unit may comprise an image sensor with at least one pixelgroup, preferably a plurality of pixel groups, wherein each of the pixelgroups comprises at least one first pixel, preferably a plurality offirst pixels, configured to detect light from the first spectral range,and at least one second pixel, preferably a plurality of second pixels,configured to detect light from the second spectral range. The pixelgroups may be arranged in particular in a plurality of rows of aplurality of pixel groups. The pixel groups may be arranged in acoplanar manner, in particular on a joint sensor chip. An opticalfilter, in particular a bandpass filter, for transmitting the particularspectral range or a sub-range of the particular spectral range may bearranged on each of the pixels.

Light from the first and second spectral range may be detectedsimultaneously in a particularly simple manner by means of an imagesensor comprising pixel groups of the described kind, and a spatiallyand spectrally resolved image dataset may be generated, which then maybe divided by the image processing unit into the first and second pixeldataset.

By appropriate choice of the number of pixels in each pixel group and/orthe optical filters arranged on the pixels of each pixel group, a highspectral resolution may be attained, and, at the same time, a high timeresolution is attainable by means of the simultaneous detection.

In the case of an image sensor having pixel groups of the kind describedabove, it may be provided that each pixel group contains at least threepixels, wherein at least two pixels of each pixel group are first pixelsas defined above and the remaining pixel or pixels are second pixels asdefined above.

The first pixels may be disjoint from the second pixels, i. e., no firstpixel is simultaneously a second pixel. One or more, but not all, of thefirst pixels may simultaneously be second pixels (for instance, in thecase of overlapping first and second spectral ranges).

If each of the pixel groups contains a plurality of first pixels and/ora plurality of second pixels, different ones of the first and/or secondpixels, respectively, may be configured to detect light from differentsub-ranges of the first and/or second spectral range, respectively. Inthis way, spectrally resolved detection of light from the first and/orsecond spectral range may be enabled.

The pixels of each pixel group may be arranged in a coplanar manner. Thepixels of each pixel group may be arranged in one or more rows of one ormore pixels per row. For example, each pixel group may comprise a row ofthree pixels or three rows of one pixel per row. In this example theimage sensor may be a conventional or adapted RGB sensor. In a furtherexample each pixel group may comprise n rows of n pixels per row (withn=1, 2, 3, . . . ).

A particularly high spectral resolution (but with reduced timeresolution) may also be attained in another way. For example, theimaging device may be configured for the sequential scanning, line byline, of the tissue area with use of a spectrally resolving detectionunit.

This may be implemented, for example, in the following way: Theillumination unit may comprise a slit aperture and a tiltable mirror forprojecting a slit-shaped illumination onto the tissue area, and theimaging unit may comprise a tiltable mirror for the stationary imagingof a region of the tissue area corresponding to the slit-shapedillumination. The imaging unit may then also comprise a spectrometerunit, for example a prism spectrometer unit or a grating spectrometerunit, which is configured for the spectral fanning out of detected lightof the region of the tissue area corresponding to the slit-shapedillumination and for imaging the spectrally fanned-out light onto animage sensor perpendicularly to the longitudinal direction of theslit-shaped illumination. By performing the tilting of the tiltablemirror and the sequential detection of images by means of the imagesensor in a temporally coordinated manner, a spatially and spectrallyresolved image dataset may then be generated.

The illumination unit and/or the imaging unit may comprise amulti-filter unit, for example a filter wheel, with a plurality ofoptical filters, wherein, by moving the multi-filter unit, each of thefilters may be introduced into a beam path of the illumination unitand/or the imaging unit. By performing the movement of the multi-filterunit and the sequential detection of images by means of the image sensorin a temporally coordinated manner, a spatially and spectrally resolvedimage dataset may then be generated. A high spectral resolution may beattained in this way as well.

A method according to the present approach for providing image-basedsupport for a medical intervention comprises the following steps:

-   -   illuminating a tissue area with light from a first spectral        range, comprising a range of visible wavelengths, and with light        from a second spectral range, which is different from the first        spectral range,    -   detecting light from the first spectral range and from the        second spectral range,    -   generating a first image of the tissue area on the basis of the        detected light from the first spectral range and a second image        of the tissue area on the basis of the detected light from the        second spectral range,    -   generating a superimposed image from the first and second image        in such a way that, in the superimposed image, on the basis of a        visual highlighting, it is possible to identify whether and        where the tissue area comprises highlight regions which are        characterized by an increased emission of light from the second        spectral range in comparison to other regions of the tissue        area.

The method, similarly to the proposed imaging device, is characterizedin that, by the generation of images (specifically the first and secondimage), different types of tissue within the tissue area may be clearlydistinguished from one another on the basis of the detected light fromdifferent spectral ranges (specifically from the first and secondspectral range) in conjunction with the visual highlighting in thesuperimposed image, thus resulting in the advantages already mentionedabove.

The method may contain further steps and/or may be refined in accordancewith the features already mentioned in conjunction with the imagingdevice.

The medical imaging device according to the present approach isparticularly well-suited for use for examining a middle-ear tissue areaby means of a method according to the invention for providingimage-based support for a medical intervention.

In the case of this use, it is possible to conclude, by means of thehighlight regions of the superimposed image, whether and where themiddle-ear tissue area comprises defectively altered epithelial tissue,in particular cholesteatoma tissue. In particular, with this use, it ispossible to benefit from the different reflection and absorptionproperties of cholesteatoma tissue and bone tissue already mentionedabove. For example, on the basis of the visual highlighting, it ispossible to check whether the cholesteatoma tissue has been fullyremoved during a surgical intervention or whether remnants of thecholesteatoma tissue still remain, which may then be identified againand removed.

BRIEF DESCRIPTION OF THE FIGURES

Embodiments of the present approach will be explained hereinafter withreference to FIG. 1 to FIG. 5 . The figures show, schematically:

FIG. 1 a basic sketch of the functioning of a medical imaging device,

FIG. 2 a a beam path of an exemplary medical imaging device,

FIG. 2 b part of the medical imaging device according to FIG. 2 a,

FIG. 3 an arrangement sketch of a further example of a medical imagingdevice,

FIG. 4 a flow diagram of a method for providing image-based support fora medical intervention,

FIG. 5 spectral absorption curves of different types of tissue.

DETAILED DESCRIPTION

Recurrent and similar features in different embodiments are provided inthe drawings with identical or similar alphanumerical reference signs.

The medical imaging device 1 sketched in FIG. 1 comprises anillumination unit 2, configured to illuminate a tissue area 3 with light4 a from a first spectral range, comprising a range of visiblewavelengths, and with light 4 b from a second spectral range, which isdifferent from the first spectral range,

-   -   an imaging unit 5, configured to detect light 4 a from the first        spectral range and to generate a first image 6 a of the tissue        area 3 on the basis of the detected light 4 a from the first        spectral range and furthermore configured to detect light 4 b        from the second spectral range and to generate a second image 6        b of the tissue area 3 on the basis of the detected light 4 b        from the second spectral range,    -   a superposition unit 7, configured to generate a superimposed        image 8 on the basis of the first and second image 6 a and 6 b        in such a way that, in the superimposed image, on the basis of a        visual highlighting 9, it is possible to identify whether and        where the tissue area 3 comprises highlight regions 10 which are        characterized by an increased emission of light 4 b from the        second spectral range in comparison to other regions 11 of the        tissue area 3.

In the superimposed image 8, both the other regions 11 and—on the basisof the visual highlighting 9—the highlight regions 10 are clearlyvisible. A distinction between different types of tissue in the tissuearea 3 in comparison to the first image 6 a and the second image 6 b isthus simplified, since in the second image 6 b the highlight regions 10,but not the other regions 11, are clearly visible, but by contrast agood overview of the tissue area 3 is provided in the first image 6 a,however the highlight regions 10 therein are not clearly distinguishablefrom the other regions 11.

The medical imaging device 1′ shown in FIG. 2 a is designed as an ENTmicroscope, which also serves as a surgical microscope. The imagingdevice 1′ is thus particularly suitable for the examination of a tissuearea 3 in the middle ear of a patient 13.

A illumination unit 2 of the imaging device 1′ comprises a broadbandlight source 14 (for example a halogen-metal halide lamp, a xenon lightsource, a broadband LED light source, or a broadband LED light sourceunit, comprising a plurality of narrow-band LED light sources) forilluminating the tissue area 3 with light 4 from a first and secondspectral range, the first spectral range corresponding to a wavelengthinterval of approximately 400 nm to 700 nm (light generated by thebroadband light source 14 outside this wavelength interval may besuppressed by means of a bandpass filter).

The second spectral range, in this example, corresponds to a wavelengthinterval of from 400 nm to 450 nm. A separate light source is notnecessary for the second spectral range, since the second spectral rangeis contained in the first spectral range, whereby the illumination unit2 is configured to simultaneously illuminate the tissue area 3 withlight 4 from the first and second spectral range.

The light 4 emitted by the broadband light source 14 may be coupled intoan objective 17 by means of a collector lens 15 and a condenser 16 inorder to illuminate the tissue area 3.

By means of the objective 17, light emitted (in particular reflected andscattered) by the tissue area 3 is detected. The light detected by meansof the objective 17 is fed to an imaging unit 5 and is divided into twobeam paths 18 a and 18 b, which correspond to the left and rightcomponent of a stereo image for three-dimensional imaging of the tissuearea 3. Alternatively, one of the beam paths 18 a and 18 b may beomitted, and the tissue area 3 may be imaged two-dimensionally.

An enlargement of the imaging may be adjusted by means of a zoom unit 19introduced into the beam paths 18 a and 18 b. By means of a tube lens20, the tissue area 3 is imaged in an intermediate image plane 21, and,from there, is imaged by means of imaging lenses 22 into an image planecorresponding to the surface of an image sensor 23. Here, the beam paths18 a and 18 b correspond to two separate regions of the image sensor 23,which correspond to the left and right components of the stereo image.The imaging lenses 22 and the image sensor 23 are part of a detectionunit 24.

The image sensor 23 has a plurality of similar pixel groups 25, whichare arranged in a plurality of rows of a plurality of pixel groups 25.One of the pixel groups 25 has been shown on an enlarged scale in FIG. 2b . The pixel group 25 comprises nine pixels 26 a-26 i, which arearranged in three rows of three pixels each (26 a-26 c, 26 d-26 f, 26g-26 i). A bandpass filter (not shown) is arranged on each of thepixels, and in this example the bandpass filters of pixels 26 a-26 ccorrespond to three sub-ranges of the second spectral range, thebandpass filters of pixels 26 d-26 i correspond to sub-ranges of thefirst spectral range lying outside the second spectral range, andtherefore the pixels 26 a-26 i are configured together for thesimultaneous and spatially separate detection of light from the firstand second spectral range. The pixel groups may comprise a differentnumber of pixels. The first and second spectral range may be dividedacross the various pixels in a different way by means of differentcombinations of bandpass filters.

The tissue area 3 is spatially resolvable by the pixel groups 25, andthe detected light is spectrally resolvable by the individual pixels 26a-26 i of each pixel group, and, in particular, may be combined in orderto generate a first image, corresponding to light from the secondspectral range detected by means of the pixels 26 a-26 c, and a secondimage, corresponding to light from the first spectral range detected bymeans of the pixels 26 a-26 i.

The image sensor 23 is connected to a superposition unit 7 (shown inFIG. 2 a ), in this example an image processing unit, which isconfigured to process the first image (in the form of a first pixeldataset) and the second image (in the form of a second pixel dataset).In particular, the superposition unit is configured to generate asuperimposed image (in the form of a third pixel dataset) on the basisof the first and second image.

The imaging device 1′ is configured to repeatedly generate the firstimage, the second image and the superimposed image by detecting a seriesof images, comprising alternating first and second images, by means ofthe image sensor 23 and by processing the series of images to generate aseries of superimposed images. The detection unit 24, by the provisionof a correspondingly high image repetition rate of the image sensor 23,is configured for the repeated detection of images with a repetitionfrequency of at least 60 Hz. The superposition unit 7, by use of aprocessing unit with sufficient processing power, is configured togenerate the third pixel dataset with a latency of less than 50 ms.

The superposition unit 7 comprises a display unit for displaying thesuperimposed image with the visual highlighting.

The image processing unit furthermore comprises a user interaction unit,by means of which a user may choose one or more image processing optionsfor the generation of the superimposed image and one or more displayoptions for the display of the superimposed image. Alternatively, animage processing option and/or a display option may also be fixed, orother image processing options and/or display options may be selectableas described hereinafter.

The superposition unit 7 is configured to generate a highlight imageaccording to the selected image processing option, which highlight imagecomprises the visual highlighting.

The following are selectable by means of the user interaction unit asimage processing options for generation of the highlight image:generation of the highlight image by means of a threshold value for thesecond image, generation of the highlight image by means of asegmentation of the second image by means of edge detection, generationof the highlight image by means of a segmentation of the second image bymeans of a region detection (for example by means of a region-growingalgorithm), generation of the highlight image by means of a color spacetransformation (for example by replacement of wavelengths which are notclearly visible by display colors that are clearly visible), generationof the highlight image with use of a principal component analysis (PCA)and/or an independent component analysis (ICA).

Other image processing options may also be selectable alternatively oradditionally, for example use of a feature extraction and/or an objectclassification and/or a machine learning algorithm and/or a neuralnetwork.

The superposition unit 7 is configured to generate the superimposedimage by superimposing the first image with the highlight image and todisplay the superimposed image in accordance with the selected displayoptions. Alternatively, the superimposed image may also be generatableby replacing regions of the first image by the highlight image or bycorresponding regions of the highlight image.

The following are selectable by means of the user interaction unit asdisplay options: display of the visual highlighting as a color-baseddelimitation of the highlight regions (for example by means of aselectable highlight color or a selectable color assignment table,wherein the color assignment table for example may assign differenthighlight colors to different values of a quantitative feature, forexample of a quotient of pixel values of the second and first image),display of the visual highlighting as an edging of the highlightregions, marking of an absence of highlight regions by means of a symbolintroduced into the superimposed image, superposition of contour lines,which correspond to different values of a quantitative feature.

The display unit configured to display the superimposed image with thevisual highlighting may be, for example, a conventional computerdisplay, configured to display the superimposed image as atwo-dimensional display image, or a 3D display, configured to displaythe superimposed image as a stereo image and/or as an AR display, forexample a computer display usable in conjunction with 3D glasses, or 3Dglasses with built-in displays for both eyes of the user.

In a further embodiment modified in comparison to the imaging device 1′shown in FIG. 2 a only in respect of the detection unit 24 and/or thesuperposition unit 7, the imaging device (not shown) has two eyepieces.The eyepieces may be digital eyepieces (i.e. eyepieces with displays fordisplaying the above-described digital superimposed image) or opticaleyepieces.

Optical eyepieces may replace the image sensor 23 in FIG. 2 a and may beconfigured to view the superimposed image in the form of an opticalimage. In this example, the superposition unit is configured to generatethe superimposed image by alternately displaying and hiding the firstand second image, for example by alternately illuminating the tissuearea 3 with light of the first and second spectral range, the frequencyof the alternation being at least 30 Hz. The superimposed image is thencreated on the basis of the perception of the viewer, who perceives therapidly alternating first and second images not as individual images,but as a superimposed image, with the visual highlighting being createdpurely optically since the second image basically shows the highlightregions. In the case of this optical superposition, in order for thehighlight regions in the second image to appear clear and the firstimage to not be too bright in comparison to the second image, the firstimage may be softenable by means of a gray filter and/or an illuminationlevel of the illumination with light from the first and/or secondspectral range may be suitably adjustable.

The medical imaging device 1″ shown in FIG. 3 comprises an endoscope 27,which is suitable for use in different diagnostic and/or surgicalinterventions for examining a tissue area 3.

The imaging device 1″ comprises an illumination unit 2, which comprisesa plurality of first LEDs 28, configured to emit light 4 a from a firstspectral range, and a plurality of LEDs 29, configured to emit light 4 bfrom a second spectral range.

The first spectral range, in this example, corresponds to a wavelengthinterval of from approximately 400 nm to 700 nm. The second spectralrange consists of two sub-ranges which correspond to the wavelengthintervals of from 480 nm to 500 nm and from 620 nm to 660 nm. Lightgenerated by the LEDs 28, 29 outside the wavelength intervals may besuppressed, as applicable, by means of bandpass filters.

By means of this selection of the first and second spectral range, theimaging device is well suited, for example, for the examination of nervetissue, in particular for distinguishing between parotid and/or nervetissue and surrounding tissue, since parotid and nerve tissue have anincreased reflection and reduced absorption at wavelengths ofapproximately 490 nm and approximately 640 nm in comparison tosurrounding tissue.

The light 4 a, 4 b emitted by the LEDs 28, 29 may be coupled into alight channel 31 of the endoscope 27 by means of a plurality of lenslets35, a collector lens 15 and an optical waveguide 30 in order toilluminate the tissue area 3.

Light 4 a, 4 b from the first and second spectral range emitted (inparticular reflected and scattered) by the tissue area 3 is detected bymeans of an imaging unit 5, which comprises a plurality of lensesarranged in an optical channel 32 of the endoscope 27, whereby thetissue area 3 is imaged onto an image sensor 23 of a detection unit 24arranged at an end of the endoscope 27, said image sensor 23 being amonochrome CCD or CMOS sensor.

The illumination unit 2 is configured for the sequential illumination ofthe tissue area 3 with light 4 a, 4 b from the first and second spectralrange. To this end, the first and second LEDs 28, 29 are switchable onand off in alternation. Alternatively, the illumination unit may have abroadband light source and a switchable filter unit for sequentialillumination of the tissue area 3.

The imaging unit 5 is configured for sequential generation of a firstimage, on the basis of the detected light 4 a from the first spectralrange, and of a second image, on the basis of the detected light 4 bfrom the second spectral range. To this end, the alternate switching onand off of the first and second LEDs 28 is synchronized with a detectionof a series of images by means of the image sensor 23, so that the firstand second image within the series of images alternate.

The image sensor 23 is connected to a superposition unit 7, in thisexample an image processing unit, which is configured to process thefirst image (in the form of a first pixel dataset) and the second image(in the form of a second pixel dataset). In particular, thesuperposition unit is configured to generate a superimposed image on thebasis of the first and second image as described in greater detail abovein conjunction with the imaging device 1′, with the difference that thefirst image, the second image, and a superimposed image in this exampleeach comprise only one component, that is to say are two-dimensionalimages and not stereo images. The imaging device 1″ of this example,however, may also be configured alternatively for the generation ofstereo images.

The image sensor 23 may also be an RGB sensor, that is to say an imagesensor with a plurality of pixel groups which each comprise a pixel fordetecting a red component, a pixel for detecting a green component and apixel for detecting a blue component of the visible light, with one ofthe components, in particular the blue component, possibly correspondingto the second spectral range. The image sensor may also be a snapshotsensor (as described above) or another type of image sensor. Forexample, the illumination units, imaging units and/or detection units ofthe imaging devices 1′ and 1″ may also be combined with one another indifferent ways, and, accordingly, a sequential and/or simultaneousillumination with light from the first and second spectral range may becombined with a sequential and/or simultaneous and/or spatially separateand/or temporally separate detection of light from the first and secondspectral range.

An exemplary method for providing image-based support for a medicalintervention will be described hereinafter with reference to FIG. 4 .

First method steps S1 a and S1 b comprise an illumination of a tissuearea with light from a first spectral range, comprising a range ofvisible wavelengths (step S1 a), and with light from a second spectralrange, which is different from the first spectral range (step S1 b).Steps S1 a and S1 b may be performed simultaneously or sequentially.

Second method steps S2 a and S2 b, which likewise may be performedsimultaneously or sequentially, comprise a detection of light from thefirst spectral range (step S2 a) and from the second spectral range (S2b).

Third method steps S3 a and S3 b comprise a generation of a first imageof the tissue area on the basis of the detected light from the firstspectral range (step S3 a) and of a second image of the tissue area onthe basis of the detected light from the second spectral range (step S3b). Steps S3 a and S3 b, again, may be performed simultaneously orsequentially.

A fourth method step S4 comprises a generation of a superimposed imagefrom the first and second image in such a way that, in the superimposedimage, on the basis of a visual highlighting, it is possible to identifywhether and where the tissue area comprises highlight regions which arecharacterized by an increased emission of light from the second spectralrange in comparison to other regions of the tissue area.

The method may contain further steps and/or may be refined in accordancewith the features already described in conjunction with the medicalimaging devices 1, 1′,1″ and in the claims and the rest of thedescription.

As already indicated in conjunction with the imaging device 1′ accordingto FIG. 2 a /FIG. 2B, such an imaging device may be used in thedescribed method in order to examine a middle-ear tissue area, and, bymeans of the highlight regions of the superimposed image, it is possibleto conclude whether and where the middle-ear tissue area comprisesdefectively altered epithelial tissue, in particular cholesteatomatissue.

This use is based on the knowledge of measured spectral properties ofcholesteatoma tissue and bone tissue which are shown in FIG. 5 .

The curve 33 (solid line) shows an absorption coefficient p_(a) of bonetissue of the middle ear (averaged over a number of samples) determinedby means of a measurement using a double-beam spectrometer in units ofmm⁻¹ as a function of a light wavelength λ in units of nm. The curve 34(dashed line) shows an absorption coefficient p_(a) of cholesteatomatissue of the middle ear (averaged over a number of samples) determinedby means of a measurement using a double-beam spectrometer in units ofmm⁻¹ as a function of a light wavelength λ in units of nm.

On the basis of the curves 33 and 34 it is clear that cholesteatomatissue has an increased reflection and reduced absorption of, inparticular, near-ultraviolet light, blue light and blue-green light incomparison to surrounding bone tissue in the middle ear. Both curves arepractically congruent below approximately 310 nm. The wavelength rangeof increased absorption by cholesteatoma tissue starts aboveapproximately 310 nm and extends on the whole to approximately 600 nm,although it should be noted that the local absorption maxima A (around400 nm) and B (double maximum around 550 nm) occur on account ofspectral properties in the blood remaining in the bone tissue samples,and therefore the spectral differences between cholesteatoma tissue andthe actual bone tissue approach one another again already below 600 nmwith increasing wavelength. Above 600 nm, the two curves are againpractically congruent.

Therefore, a selection of the second spectral range so that itcorresponds approximately to the wavelength interval of from 370 nm to480 nm or one or more sub-intervals thereof is particularly well suitedfor a distinction between the two types of tissue (cholesteatoma tissueand bone tissue), since in this range the difference between theabsorption of the two types of tissue is almost one order of magnitude.For the specified use, a selection of the first and second spectralrange as for the medical imaging device 1′ (see description of FIG. 2 aand FIG. 2 b ) is thus very suitable, and such a selection of thespectral ranges may also be provided for the imaging device 1″ or anyother embodiment of the imaging device according to the invention.

List of Reference Signs

1, 1′,1″ imaging device

2 illumination unit

3 tissue area

4, 4 a, 4 b light

5 imaging unit

6 a first image

6 b second image

7 superposition unit

8 superimposed image

9 visual highlighting

10 highlight region

11 other region

13 patient

14 broadband light source

15 collector lens

16 condenser

17 objective

18 a, 18 b beam paths

19 zoom unit

20 tube lens

21 intermediate image plane

22 imaging lens

23 image sensor

24 detection unit

25 pixel group

26 a-26 i pixels

27 endoscope

28 first LED

29 second LED

30 light guide

31 light channel

32 optical channel

33 absorption coefficient of bone tissue

34 absorption coefficient of cholesteatoma tissue

35 lenslets

A, B absorption maxima

S1 a-S4 method steps

The invention claimed is:
 1. A medical imaging device, comprising: anillumination unit configured to illuminate a middle-ear tissue area withlight from a first spectral range, comprising a range of visiblewavelengths, and with light from a second spectral range, which isdifferent from the first spectral range; an imaging unit configured todetect light from the first spectral range and to generate a first imageof the middle-ear tissue area based on the detected light from the firstspectral range, wherein the first spectral range includes at least onesubrange of visible wavelengths greater than 450 nm, and wherein theimaging unit is furthermore configured to detect light from the secondspectral range and to generate a second image of the middle-ear tissuearea based on the detected light from the second spectral range, whereinthe second spectral range includes at least one subrange of wavelengthsbetween 350 nm and 500 nm, and wherein the second spectral range omitsat least one subrange of wavelengths smaller than 380 nm; and asuperposition unit configured to generate a superimposed image using onthe first and second image, wherein the superimposed image, on the basisof includes a visual highlighting, to identify whether and where themiddle-ear tissue area comprises one or more highlighted regionsincluding cholesteatoma tissue, wherein the one or more highlightedregions including cholesteatoma tissue include an increased emission oflight from the second spectral range in comparison to other regions ofthe middle-ear tissue area including bone tissue to differentiate thecholesteatoma tissue from the bone tissue.
 2. The medical imaging deviceaccording to claim 1, wherein the second spectral range omits at leastone subrange of wavelengths greater than 450 nm.
 3. The medical imagingdevice according to claim 1, wherein the illumination unit is configuredfor a sequential illumination of the middle-ear tissue area with lightfrom the first and second spectral range, and/or wherein the imagingunit is configured for a sequential generation of the first and secondimage.
 4. The medical imaging device according to claim 1, wherein theillumination unit is configured for a simultaneous illumination of themiddle-ear tissue area with light from the first and second spectralrange, and/or wherein the imaging unit is configured for a simultaneousgeneration of the first and second image, wherein the first and secondimage are spatially separated by means of at least one optical filter.5. The medical imaging device according to claim 1, wherein thesuperposition unit is configured to generate the superimposed image byat least one of alternate generation, display, hiding, or refreshing ofthe first and second image, wherein a frequency of at least one of thealternation between the first and second image, the refreshing of thefirst or second image, generation of the first or second image, displayof the first or second image, hiding of the first or second image is atleast 30 Hz.
 6. The medical imaging device according to claim 1, whereinthe medical imaging device is or comprises an ENT microscope, a surgicalmicroscope, or an endoscope.
 7. The medical imaging device according toclaim 1, furthermore comprising: a detection unit configured to detect afirst pixel dataset, corresponding to the first image generated by theimaging unit, and a second pixel dataset, corresponding to the secondimage generated by the imaging unit, wherein the detection unit includesat least one image sensor configured for spatially resolved detection ofat least one of the first pixel dataset or the second pixel dataset, andwherein the superposition unit is or comprises an image processing unitwhich is configured to generate a third pixel dataset, corresponding tothe superimposed image, based the first and second pixel dataset.
 8. Themedical imaging device according to claim 7, wherein the detection unitis configured for a repeated detection of images with a repetitionfrequency of at least 30 Hz, preferably at least 60 Hz, and/or whereinthe image processing unit is configured to generate the third pixeldataset with a latency of less than 100 ms.
 9. The medical imagingdevice according to claim 7, wherein the detection unit comprises animage sensor with a plurality of pixel groups and each of the pixelgroups comprises at least one first pixel, configured to detect lightfrom the first spectral range, and at least one second pixel, configuredto detect light from the second spectral range.
 10. The medical imagingdevice according to claim 9, wherein the pixel groups are arranged, in acoplanar manner, in multiple rows of multiple pixel groups per row, inparticular on a joint sensor chip.
 11. The medical imaging deviceaccording to claim 7, wherein the image processing unit is configured togenerate the visual highlighting by at least one of: means of athreshold value for the second image, means of a segmentation of thesecond image, means of a color space transformation, a featureextraction, an object classification, a pattern recognition, a principalcomponent analysis, or an independent component analysis.
 12. Themedical imaging device according to claim 1, wherein the first spectralrange omits at least one subrange of the second spectral range.
 13. Amethod for providing image-based support for a medical interventioncomprising the following steps: illuminating a middle-ear tissue areawith light from a first spectral range, comprising a range of visiblewavelengths, and with light from a second spectral range, which isdifferent from the first spectral range wherein the first spectral rangeincludes at least one subrange of visible wavelengths greater than 450nm, wherein the second spectral range includes at least one subrange ofwavelengths between 350 nm and 500 nm, and wherein the second spectralrange omits at least one subrange of wavelengths smaller than 380 nm;detecting light from the first spectral range and from the secondspectral range; generating a first image of the middle-ear tissue areabased on the detected light from the first spectral range and a secondimage of the middle-ear tissue area based on the detected light from thesecond spectral range; generating a superimposed image from the firstand second image in such a way that, in the superimposed image, on thebasis of a visual highlighting, it is possible to identify whether andwhere the middle-ear tissue area comprises highlight regions which arecharacterized by an increased emission of light from the second spectralrange in comparison to other regions of the tissue area, and examiningthe middle-ear tissue to differentiate cholesteatoma tissue from bonetissue.
 14. The method according to claim 13, wherein the first spectralrange omits at least one subrange of the second spectral range.